Section: New Results

Observing and Modeling Awareness and Expertise During Problem Solving

Participants: Thomas Guntz, Dominique Vaufreydaz, James Crowley, Philippe Dessus, Raffaella Balzarini

We have constructed an instrument for the capture and interpretation of multi-modal signals of humans engaged in solving challenging problems. Our instrument captures eye gaze, fixation, body posture and facial expressions and other physiological signals from humans engaged in interactive tasks on a touch screen.

As a pilot study, we have observed the eye gaze, fixation, body posture and facial expressions for 21 chess experts engaged in solving chess problems. We are currently exploring how such recordings can be used to estimate a subjects awareness of the current situation, and ability to respond effectively to challenging tasks.